Statistical models: Conventional, penalized and hierarchical likelihood

Statistical models: Conventional, penalized and hierarchical likelihood

We give an overview of statistical models and likelihood, together with two of its variants: penalized and hierarchical likelihood. The Kullback-Leibler divergence is referred to repeatedly in the literature, for defining the misspecification risk of a model and for grounding the likelihood and the likelihood cross-validation, which can be used for choosing weights in penalized likelihood. Fami...

A ug 2 00 8 Statistical models , likelihood , penalized likelihood and hierarchical likelihood

We give an overview of statistical models and likelihood, together with two of its variants: penalized and hierarchical likelihood. The Kullback-Leibler divergence is referred to repeatedly, for defining the misspecification risk of a model, for grounding the likelihood and the likelihood crossvalidation which can be used for choosing weights in penalized likelihood. Families of penalized likel...

Penalized Likelihood Estimation in Gaussian Mixture Models

Penalized Least Squares and Penalized Likelihood

where pλ(·) is the penalty function. Best subset selection corresponds to pλ(t) = (λ/2)I(t 6= 0). If we take pλ(t) = λ|t|, then (1.2) becomes the Lasso problem (1.1). Setting pλ(t) = at + (1 − a)|t| with 0 ≤ a ≤ 1 results in the method of elastic net. With pλ(t) = |t| for some 0 < q ≤ 2, it is called bridge regression, which includes the ridge regression as a special case when q = 2. Some penal...

Penalized Likelihood Density Estimation :